Papers with masked LMs
Retrieval-based Language Models and Applications (2023.acl-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we will provide a comprehensive overview of retrieval-based language models. |
| Approach: | This tutorial will provide a comprehensive overview of recent advances in retrieval-based language models. |
| Outcome: | This tutorial will provide a comprehensive overview of recent advances in retrieval-based language models. |
Boosting Low-Resource Biomedical QA via Entity-Aware Masking Strategies (2021.eacl-main)
Copied to clipboard
| Challenge: | Biomedical question-answering (QA) provides users with high-quality information from a vast scientific literature. |
| Approach: | They propose to use a biomedical entity-aware masking strategy to fine-tune masked language models to their domains. |
| Outcome: | The proposed approach is an adaptation process for masked LMs, not memory or components. |
Contextual Distortion Reveals Constituency: Masked Language Models are Implicit Parsers (2023.acl-long)
Copied to clipboard
| Challenge: | a novel chart-based method for extracting parse trees from masked language models is proposed . a graph-based approach can be used to extract parser trees without training separate parsers . |
| Approach: | They propose a chart-based method for extracting parse trees from masked language models . they use a set of perturbations motivated by the linguistic concept of constituency tests to score each span . |
| Outcome: | The proposed method outperforms state-of-the-art methods on english with masked LMs and in multilingual settings. |
A New Formulation of Zipf’s Meaning-Frequency Law through Contextual Diversity (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies have examined Zipf's meaning-frequency law as a relationship between word frequency and the number of meanings based on contextualized word vectors . |
| Approach: | They propose to use word frequency as a relationship between word frequency and contextual diversity to examine Zipf's meaning-frequency law for a wider variety of words and corpora than previous studies have shown. |
| Outcome: | The proposed formulation gives a new interpretation of Zipf's meaning-frequency law and enables us to examine it for a wider variety of words and corpora than previous studies have shown. |
Can Machines Resonate with Humans? Evaluating the Emotional and Empathic Comprehension of LMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Empathy plays a pivotal role in fostering prosocial behavior, often triggered by the sharing of personal experiences through narratives. |
| Approach: | They propose to use contrastive learning with masked LMs and supervised fine-tuning with large language models to improve empathy understanding in NLP models. |
| Outcome: | The proposed methods show that there is low agreement among annotators and that cultural differences are a factor in their interpretation of empathy. |
-Stance: A Large-Scale Real World Dataset of Stances in Legal Argumentation (2025.acl-long)
Copied to clipboard
| Challenge: | Current tools for legal argument reasoning do not support this task. |
| Approach: | They propose to use a large-scale dataset to facilitate work on the legal argument stance classification task by evaluating whether a case summary strengthens or weakens a legal argument. |
| Outcome: | The proposed dataset is used to facilitate work on the legal argument stance classification task, which involves assessing whether a case summary strengthens or weakens a legal argument (polarity) and to what extent (intensity). |